The present disclosure relates generally to compression of three-dimensional image data and in particular to dynamic local temporal-consistent textured mesh compression.
Traditionally, digital video data is organized as a sequence of images, or frames, where the content of each frame defines a two-dimensional (2D) array of pixels to be presented on a display device. Numerous techniques have been developed to compress such data for efficient storage and transmission. Pixel-based image data is useful for some forms of video presentation; however, it is not well suited to virtual reality (VR), augmented reality (AR), or other immersive video experiences where the viewer becomes part of the scene.
Free-viewpoint video (FVV) can support a more immersive video experience. Rather than representing a frame as a two-dimensional array of pixels, FVV data typically represents each frame as a three-dimensional (3D) scene that includes one or more three-dimensional geometric objects. Within each frame, each object may have a position, and the objects may change position from one frame to the next. Further, some are all of the objects may deform (i.e., change shape). Objects can be represented using triangle meshes, as is conventional in the field of computer graphics, and the vertices of the mesh may move relative to each other to represent the changing shape of the object. Textures can be associated with the triangle meshes using conventional texture mapping techniques. To generate a video presentation using FVV, a user typically interacts with a video-rendering device (e.g., a computer) to determine a viewpoint, and the video-rendering device renders an image from that viewpoint using the object geometry of the frame. The viewpoint can be static or dynamic, depending on the application, and each frame is rendered from the appropriate viewpoint.
FVV, however, is extremely data-intensive, requiring the storage and/or transmission of object and texture data for each frame (as opposed to just a pixel array for conventional video data). This can impair performance and/or limit the contexts in which FVV can be used.